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Abstract 

Conventional  vision  systems  and  algorithms  assume  the  camera  to  have  a  single  viewpoint.  However, 
sensors  need  not  always  maintain  a  single  viewpoint.  For  instance,  an  incorrectly  aligned  system  could 
cause  non-single  viewpoints.  Also,  systems  could  be  designed  to  specifically  deviate  from  a  single  view¬ 
point  to  trade-off  image  characteristics  such  as  resolution  and  field  of  view.  In  these  cases,  a  locus  of 
viewpoints  is  formed,  called  a  caustic.  In  this  paper,  we  present  an  in-depth  analysis  of  the  viewpoint 
loci  for  catadioptric  cameras  with  conic  reflectors.  Properties  of  these  viewpoint  loci  with  regard  to 
field  of  view,  resolution  and  other  geometric  properties  are  presented.  In  addition,  we  present  a  simple 
technique  to  calibrate  such  non-single  viewpoint  catadioptric  cameras  and  estimate  their  viewpoint  loci 
(caustics)  from  known  camera  motion. 
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1  Introduction 


Traditionally,  imaging  systems  have  been  designed  to  maintain  a  single  viewpoint.  In  other  words, 
all  the  rays  of  light  entering  the  camera  intersect  at  a  single  point,  called  the  effective  pinhole.  This 
pinhole  model  is  extensively  used  in  many  vision  algorithms.  The  single  viewpoint  is  not  limited  to  lens 
based  (dioptric)  cameras  alone.  Catadioptric  (combination  of  lens  and  mirrors)  cameras  have  also  been 
designed  with  the  aim  of  maintaining  a  single  effective  viewpoint. 

Single  viewpoint  catadioptric  cameras  include  multi-sensor  planar  mirror  systems  for  panoramic  imag¬ 
ing  [20]  as  well  as  for  stereo  applications  [12].  Curved  mirrors  have  also  been  used  in  the  past.  For 
instance,  hyperbolic  [25,  28]  and  parabolic  [21,  1,  24]  have  already  been  shown  to  produce  single  view¬ 
point  catadioptric  sensors.  These  systems  consist  of  a  perspective  or  telecentric1  lens  and  a  reflector. 
The  pinhole  (entrance  pupil)  of  the  lens  is  positioned  at  one  of  the  focal  points  of  the  reflector,  making 
the  other  focal  point  the  effective  viewpoint.  However,  such  systems  require  precise  assembly  of  the 
imaging  components,  failing  which  the  viewpoint  deviates  from  a  single  point. 

Cameras  need  not  always  meet  the  single  viewpoint  constraint.  It  is  known  that  relaxing  this  constraint, 
gives  greater  flexibility  in  designing  imaging  systems.  Thus,  if  we  allow  for  deviations  from  a  single 
viewpoint  we  could  possibly  trade-off  image  characteristics  such  as  field  of  view  and  spatial  resolution. 
Catadioptric  cameras  not  having  a  single  view  point  include:  the  spherical  and  conical  reflector  based 
designs  [16,  27,  26,  3,  19,  9,  4,  10].  Chahl  and  Srinivasan  developed  a  specially  shaped  reflector  in  [8] 
so  as  that  the  visible  solid  angle  at  each  pixel  was  the  same.  In  [15],  Hicks  and  Bajcsy  developed  a  wide 
angle  perspective  projection  camera  system  also  using  specially  shaped  reflectors.  In  [22],  a  conical 
mirror  system  was  proposed  to  capture  a  high  resolution  360  x  360  degree  stereo  panorama.  All  these 
imaging  systems  do  not  maintain  a  single  viewpoint.  However,  the  images  acquired  by  such  imaging 
systems  were  processed  ignoring  non-single  viewpoint  effects.  Ideally,  the  imaging  geometry  must  be 
understood  in  order  to  correctly  process  the  acquired  images. 

When  an  imaging  system  does  not  maintain  a  single  viewpoint,  a  locus  of  viewpoints  in  three  dimensions 

is  formed,  called  a  caustic  [5].  For  dioptric  systems  this  is  termed  as  a  diacaustic  and  for  catadioptric 
'A  telecentric  lens  yields  an  orthographic  projection  of  the  scene  onto  the  image  detector. 


systems  as  a  catacaustic.  The  caustic  represents  the  envelope  of  all  incoming  scene  rays  which  are 
eventually  imaged.  Each  pixel  in  the  image  maps  to  a  point  on  the  caustic  surface.  Also,  every  point 
on  the  caustic  maps  to  an  unique  light  ray  from  the  scene  which  eventually  gets  imaged.  Thus,  caustics 
completely  describe  the  geometry  of  an  imaging  system. 

Certain  applications  demand  the  sensor  to  have  a  specific  viewpoint  surface.  In  such  cases,  one  can 
imagine  designing  specific  imaging  systems  keeping  this  desired  viewpoint  locus  in  mind.  For  instance, 
in  [1]  the  caustic  was  restricted  to  a  single  point.  In  [23]  a  stereo  sensor  was  designed  by  constraining 
the  caustic  of  the  camera  to  be  a  circle.  Also,  sensors  have  been  designed  to  provide  near-perspective 
projection  for  a  given  plane  in  the  scene  (see[15]). 

In  this  paper,  we  study  caustics  of  conic  catadioptric  systems.  By  this  we  mean  that  the  profile  of 
the  mirror  is  a  conic  section.  Conic  reflector  based  cameras  are  widely  used  in  vision  as  well  as  in 
astronomical  applications.  Also,  most  surfaces  can  be  locally  approximated  by  quadrics.  It  is  therefore 
interesting  to  analyze  the  viewpoint  loci  of  such  imaging  systems.  In  particular  we  raise  the  following 
two  questions: 

•  What  happens  to  the  viewpoint  locus  when  a  system  deviates  from  a  single  viewpoint?  Note  that  in 
spite  of  using  a  perspective  lens,  the  catadioptric  system  need  not  have  a  single  viewpoint.  Instead, 
the  viewpoint  locus  is  described  by  a  three  dimensional  caustic  surface. 

•  Is  there  a  simple  way  to  estimate  the  viewpoint  loci  for  such  non-single  viewpoint  systems?  Note 
that  in  spite  of  using  a  perspective  lens  and  reflector,  the  system  as  a  whole  need  not  maintain 
a  single  viewpoint.  Calibration  entails  finding  the  map  between  every  pixel  in  the  image  and  its 
corresponding  viewpoint. 

We  begin  by  deriving  the  caustic  surface  for  a  family  of  conic  catadioptric  systems.  By  careful  pa¬ 
rameterization  of  the  reflector  profile,  we  present  a  simple  derivation  of  the  viewpoint  locus  as  a  three- 
parameter  family  of  curves.  These  caustic  surfaces  are  then  analyzed  for  their  effects  on  resolution,  field 
of  view,  and  geometric  singularities.  It  should  be  noted  that,  in  our  analysis  we  consider  the  principal 
rays  passing  through  the  entrance  pupil  of  the  camera  lens  to  describe  the  geometry  of  the  sensor.  Sec- 


ondary  effects  such  as  blurring  (due  to  bundles  of  rays)  do  not  interfere  with  the  geometric  analysis  we 
present.  Finally,  we  present  a  simple  self-calibration  technique  to  estimate  the  parameters  that  describe 
the  caustic  surface  numerically  for  a  catadioptric  camera  using  known  camera  motion.  Although  every 
pixel  has  an  unique  viewpoint,  calibration  of  such  non-single  viewpoint  sensors  is  reduced  to  estimating 
only  a  handful  of  parameters. 

2  Caustics:  Loci  of  Viewpoints 

When  a  light  ray  interacts  with  either  a  reflective  or  refractive  interface,  it  may  bend  and  thus  alter  its 
path.  The  envelope  of  these  reflected  or  refracted  rays  is  called  the  caustic  surface  [14,  5].  Caustics 
formed  by  reflecting  elements  are  called  catacaustics  and  those  by  refractive  elements  are  called  dia- 
caustics.  Figure  1  illustrates  caustics  formed  by  (a)  reflection  by  a  metal  ring  and  (b)  refraction  through  a 
transparent  sphere.  Near  the  envelope  surface  (caustic),  the  rays  of  light  bunch  up  together,  thus  forming 
bright  patterns  as  seen  in  the  images.  Henceforth,  we  use  the  term  caustic  to  mean  both  the  catacaustic 
as  well  as  the  diacaustic. 

With  respect  to  imaging  devices,  caustics  represent  their  loci  of  viewpoints.  The  single  viewpoint  is  a 
degenerate  case  of  a  point  caustic.  Each  point  on  the  caustic  surface  represents  the  three-dimensional 
position  of  a  viewpoint  and  its  viewing  direction.  Thus,  the  caustic  completely  describes  the  geometry  of 
the  camera.  One  can  represent  compound  imaging  systems  consisting  of  multiple  imaging  components, 
such  as  lenses  and  reflectors,  by  simply  their  effective  caustics. 

2.1  Computing  the  Caustic 

We  now  study  caustics  of  catadioptric  imaging  systems  consisting  of  a  perspective  or  telecentric  lens 
and  a  single  reflector  whose  profile  is  a  conic  section.  Although,  we  use  the  perspective  or  telecentric 
(orthographic  projection)  lens,  the  imaging  system  as  a  whole  (including  the  reflector)  need  not  maintain 
a  single  viewpoint. 

We  first  present  caustics  of  rotationally  symmetric  catadioptric  systems,  where  the  entrance  pupil  of  the 
lens  is  located  along  the  axis  of  symmetry  of  the  reflector  (see  Fig. 2)  at  a  distance  d  from  the  origin  O. 
Telecentric  lenses  are  modeled  by  taking  the  limit  d  —>  oo,i.e.  modeling  an  orthographic  projection. 


(a)  (b) 


Figure  1 :  The  caustics  formed  due  to  light  interacting  with  (a)  a  metal  ring  and  (b)  a  spherical  refractive  element. 
Bright  patterns  of  light  on  the  table  illustrate  a  section  of  the  caustic  surface  formed.  These  bright  patterns  are 
due  to  the  close  bunching  together  of  light  rays  near  the  caustic’s  surface.  The  caustics  formed  are  due  to  light 
interacting  with  (a)  a  metal  ring  and  (b)  a  spherical  refractive  element.  Adapted  from  [17] 

The  reflector  profile  is  also  defined  in  this  coordinate  frame.  Since  the  system  is  rotationally  symmetric, 
analysis  is  possible  in  two  dimensions  (a  vertical  cross  section  of  the  imaging  system).  However,  for 
asymmetric  systems,  we  will  need  to  derive  caustics  in  three  dimensions  as  shall  be  shown. 

Many  techniques  have  been  proposed  to  derive  the  caustics  of  reflecting  and  refractive  systems  including 
ones  based  on  local  conic  approximations  [6]  and  the  Jacobian  method  [7].  In  [6],  the  central  idea  lies 
in  fact  that  for  conic  sections  the  light  rays  emanating  from  one  of  the  focal  points  all  meet  at  the 
second  focal  point.  To  estimate  the  caustic  point  corresponding  to  a  point  on  an  arbitrary  curve,  we  first 
approximate  the  curve  locally  by  a  conic  section  such  that  one  of  the  foci  lies  at  the  entrance  pupil  of  the 
lens.  Thus,  the  second  focal  point  (derived  analytically)  is  the  corresponding  point  on  the  caustic  curve. 
However,  this  approach  does  not  extend  to  three  dimensions  as  is  required  in  asymmetric  systems.  We 
therefore  derive  the  caustic  surface  based  on  the  Jacobian  approach  [7],  since  it  easily  extends  to  three 
dimensions. 


In  analysing  these  imaging  systems  we  use  the  framework  of  geometric  optics.  Thus,  only  the  chief  rays 


Image  of  world  point 


Figure  2:  An  imaging  system  consisting  of  a  conic  reflector  and  a  perspective  lens  based  camera.  The  entrance 
pupil  of  the  lens  is  located  at  a  distance  d  from  the  origin,  along  the  axis  of  symmetry  of  the  conic  section.  The 
reflector  profile  is  also  defined  in  this  coordinate  frame.  A  light  ray  from  the  scene  reflects  off  the  reflector  surface 
and  is  imaged  after  passing  through  the  entry  pupil.  A  telecentric  lens  is  modeled  by  taking  the  limit  d  oo. 
Note  that  this  imaging  system  can  have  a  non-single  viewpoint  in  spite  of  using  a  perspective  lens. 

or  princilpal  rays  are  considered.  Although  every  pixl  is  associated  with  a  bundle  of  rays  from  the  scene, 
only  the  principal  ray  are  important  from  a  geometric  standpoint  and  will  be  used  in  our  analysis.  In 


contrast,  budles  of  rays  are  needed  to  analyse  secondary  effects  such  as  blurring  which  is  beyond  the 
scope  of  this  paper. 

2.1.1  The  Reflector  Surface 

Parameterization  of  the  reflector  surface  is  an  essential  step  towards  computing  the  caustic  surface  ana¬ 
lytically.  Indeed,  whatever  the  parameterization  for  the  reflector,  a  solution  for  the  caustic  surface  exists. 
We  found  that  standard  parameterizations  used  for  conics  lead  to  complicated  solutions  that  are  difficult 
to  analyze.  In  contrast,  the  following  generic  parameterization  yields  a  simple  solution  for  the  caustic 
surface.  Referring  to  Fig. 2,  we  define: 

z{t)  =  t  (1) 

7  (t)  =  \J  (e2  —  l)t2  +  2  pt  —  p 2 

where,  e  is  the  eccentricity  and  p  the  focus  of  the  conic  section.  This  represents  elliptic  (e  <  1),  parabolic 
(e  =  1)  and  hyperbolic  (e  >  1)  reflectors.  The  vertex  of  the  reflector  is  given  by  The  T-axis  is  the 
directrix  of  the  conic  reflector.  A  point  on  the  reflector  surface  is  then  Sr(t)  =  [z(t).  7 (£)].  Although, 
this  parameterization  has  a  singularity  for  spherical  reflectors,  it  makes  it  easier  to  analyse  and  describe 
the  various  geometrical  properties  of  the  family  of  viewpoint  loci.  In  Appendix  A,  we  derive  caustics 
using  a  more  generic  parameterization  which  includes  the  sphere. 

2.1.2  The  Rotationally  Symmetric  Caustic  Surface 

From  Fig. 2,  the  vector  along  the  reflected  ray  (entering  the  lens  pupil)  is  given  by: 

Vr(t)  =  [t  +  d,yj ( e 2  -  l)t2  +  2 pt  -  p2}.  (2) 

Since  we  know  the  geometry  of  the  reflector,  its  surface  normal  Nr(t)  can  be  derived  analytically.  We 
then  derive  the  pencil  of  incident  rays  Vj(t)  upon  reflection  as: 

Vj (t)  =  Vr(f)  —  2Nr(f)  (Nr(t)  •  Vr(t)) .  (3) 

A  point  along  the  incident  ray  (along  V;(t))  parameterized  by  its  distance  r  from  the  point  of  reflection 
Sr(t)  is  then  given  by:  Sr(t)  +  r  ■  Vi(f).  The  caustic  is  tangential  to  the  ray  along  Vi(f)  and  hence  for 


z 

Figure  3:  Section  of  the  reflector  showing  incident  rays  from  the  scene  reflected  into  the  lens.  The  incident  ray 
Vi(i)  is  tangential  to  the  caustic  surface.  The  distance  of  the  point  on  the  caustic  from  the  point  of  reflection  is 
denoted  by  rc.  At  the  caustic  point,  if  we  travel  infinitesimally  along  V;(t),  we  would  also  move  from  one  ray 
onto  the  next.  This  is  due  to  the  close  bunching  of  extended  incoming  rays  at  the  caustic  surface. 

some  rc  the  caustic  point  lies  at  (see  Fig.  3): 


Sr(t)  +  r c  •  Vi {t). 


(4) 


At  this  point,  the  light  rays  bunch  up  close  together  to  form  an  envelope.  Thus,  in  the  limit,  traversing 
infinitesimally  along  Vj(f)  at  the  caustic,  is  equivalent  to  traversing  from  one  ray  to  the  next. 


Therefore,  at  the  caustic  surface,  the  determinant  of  the  Jacobian  J(Sr(f)  +  rc  ■  Vr(f))  must  vanish  (see 
[7]  for  details).  Let  us  now  denote  the  Z  and  T  components  of  Sr(t)  by  Sr(t)z  and  Sr(f)7,  respectively, 
and  those  of  Vi(f)  by  Vi(f)„  and  V; (f)7.  Enforcing  the  vanishing  constraint  on  Eq.4  we  get: 


det 


=  0, 


'  [  SrW,  +  rc  ■  v,(i),  V,(i), 

J  Sr(«)7  +  >V  V,(i)7  V,(«)7 

where,  Sr(f)  =  dS^l:>  and  Vi  (t)  =  <rv^ .  Solving  for  rc  we  get: 

=  S -  Sr(t)zVj(t)7 
vi(f),vi(f)7-vi(f)7vi(f); 

For  the  class  of  conic  catadioptric  cameras,  rc  is  given  by: 

t(2p  +  t  ( e 2  -  1))  (t  (. d  (1  -  e2)  +  p)  -  p  (d  +  p)) 


(5) 


(6) 


rc(t)  = 


-•(7) 


2 p2  (p2  —  d2)  —  6 tp2  (d  +  p)  —  3 t2p  (d  +  p)  ( e 2  —  1)  —  t3  ( e 2  —  1)  (d  ( e 2  —  1)  d  —  p) 

Substituting  Eq.  (7)  in  Eq.  (4),  we  get  the  caustic  profile  for  the  family  of  rotationally  symmetric  conic 
catadioptric  cameras  as  a  three  parameter  (e,p,  d)  family  of  curves: 


= 


N, 


(8) 


N  7  , 

7C  =  — ,  where, 

J-Zr 


Nz  =  2p3(d  +  p)2  +  6(d(e2  —  1)  —  p)p2{d  +  p)t 
+3 p{d  +  p)  (d( 2  -  3e2  +  e4)  +  2 p)t2  + 
(d2(e2  —  2)(e2  —  l)2  —  <7(4  —  7e2  +  3e4)p 
+2(e4  +  e2  -  1  )p2)t3, 

=  2{d  +  p)(d  —  de2  +  p  +  e2p) 

((e2  -  l)f2  +  2 pt  -  p2)%, 

Dc  =  e2(2{d  —  p)p2(d  +  p)  +  6p2(d  +  p)t 
+3(e2  -  1  )p(d  +  p)t2  +  (e2  -  1) 

{d(e2  —  1)  —  p)t3). 


The  caustic  produced  due  to  a  telecentric  lens  and  a  conic  reflector  is  obtained  by  taking  the  limit  d  — >  oo: 

z™  =  (2p3  +  6(e2  —  l)p2t  +  3(2  —  3e2  +  eA)pt2  + 

(e2-2)(e2-l)V)/(2eV),  (9) 

7C°°  =  ((l-e2)((e2-l)t2  +  2pt-p2)l)/(eV). 

We  observe  from  Eq.(8)  that  the  caustic  surface  is  dependent  on  the  distance  d  of  the  entry  pupil  with 
respect  to  the  reflector,  the  reflector  geometry  parameterized  by  eccentricity  e,  and  the  focal  point  p. 
Since  conic  reflector  based  catadioptric  systems  also  give  single  viewpoint  imaging  systems,  we  ask:  At 
what  distance  c/0  of  the  entrance  pupil  would  the  system  produce  a  point  caustic  (single  viewpoint)  at  the 
focus  (p,  0)  of  the  reflector?  From  Eq.(8),  we  set  zc  =  p,  and  7c  =  0  and  solve  for  d0: 

e2  +  1 

do  =  p— - -•  (10) 

ez  —  1 

In  Eq.(10),  setting  e  =  1  (parabolic  reflector)  gives  d0  =  oo,  implying  the  use  of  a  telecentric  lens  as 
in  [21].  Solutions  for  elliptical  (e  <  1)  and  hyperbolic  (e  >  1)  reflectors  suggest  the  use  of  perspective 
lenses  located  at  the  focal  point  of  the  reflectors.  Thus,  using  the  caustic  surfaces  derived  above,  we  can 
describe  single  viewpoint  systems  as  a  special  case  of  the  general  solution. 

2.1.3  The  Asymmetric  Caustic  Surface 

As  noted  above,  when  the  imaging  system  is  rotationally  symmetric,  we  can  compute  its  caustic  surface 
as  a  2D  profile.  This  eases  computation  and  provides  a  simple  solution  for  the  three  dimensional  sur¬ 
face.  However,  when  the  system  is  not  rotationally  symmetric,  we  need  to  compute  the  caustic  in  three 
dimensions  .  This  occurs  when  the  entrance  pupil  is  not  along  the  axis  of  the  reflector  or  the  reflector 
itself  is  not  rotationally  symmetric.  We  now  present  the  derivation  of  the  caustic  surface  for  this  generic 
setting. 

Referring  to  Fig. 4,  let  Sr(f,  u)  be  a  point  on  the  three  dimensional  reflector,  parameterized  by  (t,  u).  Let 
P  denote  the  position  of  the  entrance  pupil  of  the  lens.  For  any  such  point  Sr(f,  u)  on  the  reflector,  the 
ray  entering  the  pinhole  can  be  derived  analytically  as: 


\r(t,u)  =  P  —  S  r(t,u). 
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Figure  4:  A  lens  based  camera  placed  off-axis  with  respect  to  a  rotationally  symmetric  conic  reflector  parameter¬ 
ized  by  (t,u).  The  viewpoint  locus  (caustic  surface)  of  such  imaging  systems  is  not  rotationally  symmetric  and 
hence  must  be  derived  in  three  dimensions.  The  light  ray  from  the  scene  V)(t ,  u)  reflects  off  the  reflector  surface 
at  Sr(f,  u)  and  is  imaged  after  passing  through  the  entrance  pupil  O  along  the  reflected  ray  Vr(t.  u). 

From  the  known  reflector  geometry  we  can  derive  the  vector  Vi  (t,  u )  along  the  incoming  light  ray  from 
the  reflection  equation  (similar  to  Eq.3). 

Again,  we  denote  the  distance  from  the  point  of  reflection  at  which  the  caustic  lies  along  V;  by  rc.  The 
caustic  surface  then  is  given  by: 


S c(t,u)  =  Sr(t,  u)  +  rc  ■  Vi(f,  u) 


(12) 


Applying  the  Jacobian  method,  and  the  vanishing  determinant  constraint,  we  get: 


et  [J  (Sr  (t, 

u)  +  r c  • 

9Vi  (t,u)x 

dSr(t,u)  x 

dt 

du 

<9Vi  (t,u)Y 

8S  r(t,u)Y 

dt 

du 

d'Vi  (t,u)z 

dSr{t,u)  z 

at. 

du 

=  0, 


where  the  X,Y  and  Z  components  of  the  vectors  are  denoted  as  S r(t,u)x,  S r(t,u)Y  and  S r(t,u)z, 
respectively.  Solving  the  above  equation,  gives  us  a  quadratic  equation  in  rc,  the  roots  of  which  give  us 
the  analytic  solution  for  rc.  The  appropriate  root  of  the  equation  can  be  substituted  in  Eq.(12)  to  derive 
the  analytic  form  of  the  caustic  in  three  dimensions. 

2.2  Examples  of  Caustic  Surfaces 


We  now  present  the  viewpoint  loci  for  typical  conic  catadioptric  systems.  Figures  5(a),  (d)  and  (g) 
illustrate  viewpoint  loci  (gray  curves)  for  a  catadioptric  sensor  consisting  of  a  perspective  lens  and  an 
elliptic,  a  parabolic  and  an  hyperbolic  reflector  (dark  curves),  respectively.  The  dotted  curves  in  Fig. 5(a) 
denote  the  part  of  the  elliptic  reflector  that  is  self-occluded  as  well  as  its  corresponding  “virtual-caustic”. 
Similarly,  Fig. 5  (b),  (e)  and  (h)  show  profiles  for  catadioptric  systems  consisting  of  a  telecentric  lens  and 
an  elliptic,  parabolic  and  hyperbolic  reflector,  respectively.  Note  that  in  Fig. 5(e)  the  caustic  degenerates 
to  a  point  as  expected. 

Figure  5(c)  is  a  three-dimensional  plot  of  the  caustic  surface  for  a  symmetric  system  consisting  of  a 
parabolic  reflector  and  a  perspective  lens.  Figure  5(f)  depicts  the  caustic  surface  in  three  dimensions  for 
an  asymmetric  (pupil  not  on  axis  of  symmetry)  catadioptric  system.  Figure  5(i)  is  the  viewpoint  locus 
for  a  catadioptric  system  including  a  hyperbolic  reflector  and  a  telecentric  lens.  Unlike  the  other  caustic 
surfaces  that  are  bounded  by  their  reflector’s  sizes,  this  caustic  surface  expands  radially. 

From  the  above  examples  it  is  clear  that  the  viewpoint  locus  can  be  potentially  large  and  at  times  as 
large  as  the  reflector  itself.  To  process  the  image  formed  by  such  sensors,  one  must  first  understand 
the  geometry  of  the  imaging  system.  The  viewpoint  locus  derived  using  the  framework  of  caustics 
completely  describes  the  geometry  of  the  imaging  system. 


(g)  (h)  (i) 


Figure  5:  Viewpoint  loci  for  conic  catadioptric  systems.  Column  one:  Caustics  for  symmetric  systems  consisting 
of  a  perspective  lens  and  an  (a)  elliptic,  (d)  parabolic  and  (g)  hyperbolic  reflector.  Column  two:  Viewpoint  loci  for 
catadioptric  systems  consisting  of  a  telecentric  lens  and  an  (b)  elliptical,  (e)  parabolic  and  (h)  hyperbolic  reflector. 
Column  three:  (c)  Caustic  surface  in  3 D  for  a  symmetric  catadioptric  system  consisting  of  a  parabolic  reflector  and 
perspective  lens,  (f)  Caustic  surface  for  an  asymmetric  catadioptric  camera  consisting  of  an  off-axis  perspective 
lens  and  a  parabolic  reflector,  (i)  Caustic  surface  for  a  telecentric  and  hyperbolic  reflector  system.  All  caustics 
were  derived  using  the  Jacobian  technique  described  in  Section  2. 
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Figure  6:  Typical  catadioptric  camera  consisting  of  a  reflector  and  a  perspective  lens.  The  caustic  has  a  singularity 
denoted  by  Zc  which  we  call  the  cusp.  Also  shown  is  the  tangent  ray  to  the  reflector  surface.  The  point  of 
tangentiality  T  is  also  a  point  on  the  caustic  surface. 

3  Properties  of  Caustic  Surfaces 

We  now  present  some  characteristic  properties  of  caustics  such  as  surface  singularities  and  field  of  view 
and  their  relevance  to  the  design  of  imaging  systems.  These  apply  to  both  convex  reflectors  and  concave 
reflectors.  Most  of  the  observations  made  are  relevant  to  rotationally  symmetric  as  well  as  asymmetric 
case  of  imaging  systems.  However,  derivations  are  shown  for  the  rotationally  symmetric  systems  for 
clarity. 

3.1  Singularities  on  the  Caustic  Surface 

As  seen  from  Fig. 6,  caustic  surfaces  have  singularities  which  we  refer  to  as  cusps.  These  correspond  to 
the  points  on  the  reflector  where  its  surface  normal  coincides  with  the  reflected  light  ray  (along  Vr(t,  u)) 
which  enters  the  entrance  pupil  of  the  lens  (see  Fig. 6).  This  constraint  in  general  is  given  by: 


Vr(f,w)  =  — N  r(t,u). 


For  rotationally  symmetric  systems  with  convex  or  concave  reflectors,  we  get: 


Vr(t)  =  — Nr(t). 


(14) 


In  such  systems,  the  cusp  lies  along  the  optical  axis.  Referring  to  Fig. 6,  we  only  need  to  compute  the  Z 
coordinate  of  the  cusp.  From  Eq.(8)  and  Eq.(14),  we  solve  for  the  Z  coordinate  of  the  cusp: 

v  _p((l  +  e){2  +  e  +  e2)d  +  2(l+e  +  e2)p) 

C  (1  +  e)  (2  (1  +  e)  (i  +  (2  +  e  +  e2)  p)  '  1  J 

3.1.1  The  Cusp  and  Rotationally  Symmetric  Systems 


We  now  discuss  the  significance  of  the  cusp  to  the  design  of  imaging  systems  whose  caustics  are  ro¬ 
tationally  symmetric.  We  pose  the  problem  of  creating  a  spherical  or  cylindrical  panorama  from  the 
image  acquired  from  a  such  non-single  viewpoint  sensor.  In  such  cases,  a  single  center  of  projection  is 
assumed.  Since  our  sensor  does  not  have  a  single  center  of  projection,  parallax  effects  are  introduced 
in  the  panorama.  In  this  setting,  how  do  we  measure  the  degree  of  parallax  errors  in  the  panorama?  To 
measure  these  errors,  we  need  the  location  of  the  virtual  center  of  projection. 

We  first  observe  that  any  incoming  light  ray,  which  eventually  gets  imaged  by  the  detector,  meets  the 
optical  axis  at  some  point.  At  the  cusp,  the  light  ray  passes  along  the  optical  axis  into  the  lens.  As  we 
move  radially  outwards,  all  incoming  light  rays  (tangent  to  the  caustic  profile)  intersect  the  optical  axis 
at  a  point  which  moves  towards  the  entrance  pupil  of  the  lens.  Finally,  at  the  very  extreme  end  of  the 
caustic  surface,  where  it  touches  the  reflector,  the  incoming  ray  grazes  the  reflector  and  meets  the  optical 
axis  at  the  entrance  pupil.  Thus,  all  the  light  rays  entering  the  imaging  system  (tangent  to  the  caustic 
surface)  intersect  the  optical  axis  between  the  entrance  pupil  and  the  cusp.  The  cusp  and  the  entrance 
pupil  therefore  bound  the  possible  locations  of  the  virtual  center  of  projection  (single  viewpoint). 

As  the  entrance  pupil  moves  closer  to  the  reflector,  the  cusp  moves  towards  the  entrance  pupil.  However, 
when  the  lens  moves  farther  from  the  reflector,  the  cusp  moves  in  the  opposite  direction.  In  Eq.(15), 
taking  the  limit  d  — >  oo,  we  find  that  the  position  of  the  cusp  converges  to: 

7oo  _  p(2  +  e(l  +  e)) 

C  2  (1  +  e) 


(16) 


Thus,  even  if  the  exact  location  of  the  entry  pupil  is  unknown,  one  can  estimate  an  upper  bound  on  the 
cusp’s  location  and  still  get  a  good  bound  on  where  to  approximate  the  viewpoint  locus  by  a  single  point. 

3.2  Caustics  and  Field  of  View 

Catadioptric  systems  consisting  of  convex  reflectors  have  a  pencil  of  rays  which  graze  the  reflector 
surface  (see  Fig.  7).  These  rays  define  the  limit  of  the  field  of  view  of  the  camera.  Also,  since  the  caustic 
touches  the  reflector  at  this  point,  it  bounds  the  size  of  the  caustic  surface. 

We  now  show  that  for  reflectors  without  points  of  inflection,  the  point  of  tangentiality  of  the  grazing  ray 
is  also  its  corresponding  caustic  point. 


Theorem  1  The  point  on  the  reflector  at  which  the  light  ray  passing  through  the  lens  ’  entrance  pupil 
pupil  grazes  the  reflector,  is  also  its  caustic  point. 


Proof:  Let  any  point  on  the  reflector  be  given  by  Sr(x,  y)  =  {x,  y ,  Z(x,  y)}.  Let  the  position  of  the 
entrance  pupil  of  the  lens  be  given  by  P  =  {U,V,W}.  The  light  ray  entering  the  pupil  is  then  given  by: 

Vi  (x,y)  =  Sr(x,y)~P  (17) 


For  the  known  reflector,  the  surface  normal  is  derived  as: 


Nr  (x,y)  = 


1  I  9Z (x,y) 2  ,  dZ (x,y) 2 
1  “r  dx  -r  dy 


•{- 


dZ(x,y)  dZ(x,y) 


dx 


dy 


,1} 


(18) 


Since  we  know  that  the  reflected  ray  along  Vr(x,  y)  grazes  the  reflector  surface,  it  must  lie  in  the  plane 
tangent  to  Sr  (x,y).  We  can  thus  select  an  appropriate  coordinate  frame  such  that  the  XY  -  plane  cor¬ 
responds  to  the  tangent  plane,  giving  r>ZgXx,y'>  =  0  and  <>Zgyy^  =  0.  Also,  since  the  pupil  P  lies  in  the 
XY -plane,  W  =  0. 


Now  using  the  method  described  in  [7]  we  compute  the  desired  Jacobian  as: 


J  (Sr (x,y)  +rc-Vr(x,y)) 


—  U  1  +rc  0 

— V  0  1  +rc 

-2rc(Vs^l  +  Ud\ gal) 


0 


(19) 


Figure  7 :  A  catadioptric  system  consisting  of  an  hyperbolic  reflector.  A  entry  pupil  at  dj  possesses  a  ray  which 
is  tangential  to  the  hyperbola  at  a  finite  point.  However,  no  light  ray  entering  the  pupil  d oo  can  be  tangential  to  the 
reflector. 

Solving  for  rc  (when  the  Jacobian  vanishes)  we  get: 

rc  =  0 

rc  =  1  (20) 

The  second  solution  rc  =  1  is  meaningless.  However,  the  right  solution  is  given  by  rc  —  0. 

This  proves  that  the  caustic  point  and  the  point  of  tangentiality  coincide.  □ 

We  showed  that  in  three  dimensions  the  grazing  ray  is  tangent  to  the  reflector  at  a  point  which  is  also 
a  point  on  the  caustic  surface.  A  similar  result  has  been  already  shown  for  caustics  of  2D  curves  with 
respect  to  a  point  in  a  plane  (see  [6]). 


3.2.1  Location  of  the  Tangent  Point 


We  now  derive  the  exact  parametric  form  for  the  location  of  the  “tangent  point”  using  the  constraint 
rc  =  0  for  the  class  of  rotationally  symmetric  conic  catadioptric  systems.  Setting  Eq.(6)  to  be  zero,  we 
solve  for  the  point  of  tangentiality  parameterized  by  tr=0: 


p(d  +  p) 
d  +  p  —  de2  ’ 


(21) 


Amongst  the  class  of  conic  reflectors,  only  hyperbolic  mirrors  do  not  always  have  a  grazing  ray  because 
the  reflector  is  asymptotic.  If  the  entrance  pupil  of  the  lens  is  placed  between  the  vertex  of  the  reflector 
and  the  point  of  intersection  between  its  two  asymptotes,  a  tangent  ray  (grazing  ray)  exists  which  touches 
the  reflector  at  a  finite  point.  In  contrast,  if  the  entrance  pupil  is  located  farther  away,  then  there  is  no 
tangent  ray  to  the  reflector  (see  Fig. 7).  The  caustic  of  such  an  imaging  system  is  also  asymptotic,  and 
approaches  but  never  touches  the  reflector  surface. 

Due  to  the  non-single  viewpoints,  perspective  (or  panoramic)  views  created  would  be  distorted  due  to 
parallax  effects.  Knowing  the  point  at  which  the  caustic  touches  the  reflector  surface,  gives  us  a  measure 
of  the  caustic’s  size.  This  in  turn  bounds  the  distortions  due  to  parallax. 


4  Need  for  computing  Caustics  in  3D 

In  the  previous  section  we  claimed  that  caustics  for  rotationally  asymmetric  imaging  systems  need  to 
be  derived  in  three  dimensions.  Further,  it  is  impossible  to  reduce  the  derivation  of  the  caustic  point 
corresponding  to  a  point  on  the  reflector  to  two  dimensions.  We  justify  these  claims  by  first  providing 
an  intuitive  reasoning  which  is  later  substantiated  with  a  real  example. 

Referring  to  Fig.  8,  incoming  light  ray  V;(f)  from  the  scene  lies  in  the  plane  II  defined  by  the  surface 
normal  Nr(f)  at  the  point  of  reflection  Sr(t)  and  reflected  ray  Vr.(7  )  entering  the  lens.  Hence,  the  caustic 
point  Sc  should  also  lie  in  this  very  plane  II.  Now,  the  plane  II  and  the  reflector  intersect  to  form  a  planar 
curve  in  3D.  If  two  dimensional  analysis  were  sufficient,  we  should  be  able  to  derive  the  caustic  point, 
from  the  planar  curve,  in  II  alone.  However,  we  further  observe  that,  the  surface  normals  along  the  curve 
need  not  lie  in  the  plane  II.  This  non-coplanarity  of  surface  normals  entails  a  complete  3D  analysis  of 


Figure  8:  An  asymmetric  catadioptric  camera  consisting  of  a  reflector  and  perspective  lens  whose  entrance  pupil 
is  located  at  O.  The  normal  at  a  point  of  reflection  Sr(T)  and  the  reflected  ray  Vr  together  define  the  plane  II  in 
which  the  incident  ray  Vi  must  lie.  The  plane  II  intersects  the  reflector  to  give  us  a  planar  curve  in  3D.  The  normal 
at  some  other  point  Sr(t")  on  the  reflector  along  this  curve  clearly  does  not  lie  in  the  plane  II.  Furthermore,  the 
normal  arc  continuous  and  thus  even  in  the  neighborhood  of  S r(t)  the  normals  are  non-coplanar.  Thus,  solving 
for  the  caustic  corresponding  to  S r(i)  using  the  planar  curve  is  erroneous. 

the  caustic  surface. 

To  illustrate  this  fact,  compare  the  two  approaches  of  deriving  the  caustic  point  for  a  point  on  a  given 
reflector.  If  the  3D  and  2D  analysis  were  equivalent,  then  the  caustic  point  should  lie  at  the  same 
distance  rc  from  the  reflector  in  both  cases.  Thus,  comparing  the  distance  rc  of  the  caustic  from  the  point 
of  reflection  is  sufficient. 

Consider  a  parabolic  reflector  and  a  perspective  lens  in  an  off-axis  configuration  as  shown  in  Fig. 9  (a). 
In  particular,  the  reflector  surface  is  defined  by: 

Sr  (x,y)  =  {x,y,-(z2  +  y2)}.  (22) 

This  represents  a  parabolic  reflector,  whose  vertex  lies  at  the  origin.  To  obtain  an  asymmetric  imaging 
system,  we  locate  the  entrance  pupil  of  the  lens  at  (1,  0, 1}  in  the  VZ-planc.  This  creates  an  asymmetric 
viewpoint  locus  as  shown  in  Fig. 9  (b). 


Figure  9:  (a)  An  asymmetric  catadioptric  camera  consisting  of  a  parabolic  reflector  and  perspective  lens  whose 
entrance  pupil  is  located  at  {1, 0, 1}.  (b)  The  corresponding  asymmetric  viewpoint  surface  computed  using  purely 
three  dimensional  analysis.  Note  the  peculiar  shape  and  pinching  at  the  cusp. 


We  now  consider  a  point  on  the  reflector  such  that  the  incidence  plane  does  not  pass  through  the  axis  of 
symmetry  of  the  reflector2.  Thus,  we  wish  to  compute  the  caustic  corresponding  to  the  point  of  reflector 
located  at  O  =  {.5,  .5,  —.5}. 

4.0.2  Deriving  the  Caustic  in  3D. 

As  suggested  in  §2,  we  derive  the  caustic  surface  using  the  vanishing  Jacobian  constraint^]  (see  Fig. 9(b)). 

For  the  selected  point  on  the  reflector  Sr(.5,  .5),  the  distance  of  the  caustic  point  from  the  point  of 
refection  (see  Eq.  (6))  is  computed  to  be: 

rc(.5,  .5)  =  -0.193897.  (23) 

2It  can  be  shown  that  if  the  incidence  plane  passes  through  the  axis  of  symmetry,  all  surface  normals  for  reflector  points 


which  are  in  this  plane  also  lie  in  the  very  same  plane. 


Figure  10:  (a)  A  section  of  the  planar  curve  formed  by  the  intersection  of  the  incident  plane  II  and  the  parabolic 
reflector,  (b)  The  curve  plotted  in  (a)  is  superimposed  on  the  reflector  to  illustrate  its  exact  nature  and  geometric 
relationship  with  the  entrance  pupil  of  the  perspective  lens. 

4.0.3  Deriving  the  Caustic  in  2D. 

As  discussed  earlier,  we  expect  the  caustic  to  lie  in  the  incidence  plane  II  defined  by  the  surface  normal 
at  the  point  of  incidence  and  the  reflected  ray  Vr  entering  the  entrance  pupil  of  the  lens.  The  incidence 
plane  II  and  the  reflector  carve  out  the  planar  curve  as  shown  in  Figure  10(a-b).  For  two  dimensional 
analysis,  we  restrict  the  caustic  computation  to  this  planar  curve. 

The  first  step  in  this  process  is  to  derive  the  planar  curve  carved  by  the  intersection  of  the  incidence  plane 
II  and  the  reflector.  We  define  the  incidence  plane  II  by  its  normal  vector: 

n  =  Vr(x,y)  x  Nr(x,y).  (24) 

We  now  determine  the  locus  of  points  which  lie  on  the  intersection  of  II  and  the  reflector.  Note  that  the 
dot  product  between  the  vector  joining  the  entrance  pupil  O  and  any  other  point  of  the  surface  Sr(x',  y') 


Figure  11:  The  planar  curve  (see  Fig.  10)  was  transformed  so  that,  the  point  of  interest  on  the  reflector,  namely 
the  point  {.5,  .5,  —.5}  now  lies  at  the  origin  and  was  rotated  appropriately  so  as  to  completely  lie  in  the  XZ-plane. 
Once  transformed,  we  can  analyze  the  caustic  of  any  point  on  this  planar  curve  simply  in  the  XZ- plane.  The 
entrance  pupil  was  also  appropriately  transformed  and  is  shown  as  a  big  black  dot. 

with  the  II  must  vanish.  This  constraint  is  given  by: 

SrO^t/O-o-ri  =  o.  (25) 

Solving  Eq.(25)  for  x',  gives  us  a  parametric  form  for  the  planar  curve  along  the  reflector  of  the  form 
(see  Figure.  10(a-b)): 

SfV)  =  {x\y'),y',zW)}.  (26) 

To  simply  computing  the  caustic,  we  now  apply  a  transformation  to  the  curve  as  well  as  the  entrance 
pupil,  so  as  to  fix  them  in  the  X,  Z-plane  as  well  as  move  the  point  on  the  reflector  to  the  origin.  Figure 
11  shows  the  resulting  planar  curve  and  the  entrance  pupil  O  after  applying  the  transformation  in  the 
A'Z-planc. 

We  now  derive  the  distance  of  the  caustic  point  corresponding  to  {0,  0}  using  the  Jacobian  technique  for 


two  dimensions  as: 


r2cD( 0)  =  -0.272784. 


(27) 


From  Eq.  (23)  and  Eq.  (27)  it  is  clear  that  the  distance  of  the  caustic  from  the  reflector  surface  is 
different  for  the  two  approaches.  The  two  dimensional  analysis  (in  the  incidence)  plane  and  the  complete 
three  dimensional  analysis  provide  different  results.  Thus  we  show  by  example  the  need  for  pure  three 
dimensional  analysis  when  deriving  caustics  for  asymmetric  imaging  systems. 


5  Resolution 


So  far  we  analysed  the  geometry  of  the  viewpoint  surface,  its  singularities  and  other  properties.  We  now 
analyse  the  effect  of  such  viewpoint  surfaces  on  resolution  characteristics  of  the  sensor.  We  derive  an 
analytic  expression  for  resolution  and  show  how  small  design  changes  impact  resolution  characteristics 
dramatically. 

In  [2],  Baker  and  Nayar  showed  that  conic  reflector  based  single  viewpoint  catadioptric  cameras  possess 
radially  increasing  resolution.  Such  imaging  systems  are  a  special  case  of  conic  catadioptric  cameras. 
We  now  extend  their  analysis  to  include  non-single  viewpoint  imaging  systems  as  well. 

5.1  Single  Reflector  with  Perspective  Lens 


Consider  an  infinitesimal  area  6 A  in  the  imaging  plane.  The  angle  subtended  by  this  region  about  the 
entrance  pupil  is  ip.  Let  this  area  image  an  infinitesimal  solid  angle  5uj  of  the  scene  (see  Fig.  12).  The 
resolution  of  the  imaging  system  is  then  defined  as:  |-j.  Thus,  smaller  the  solid  angle  visible  at  the 
infinitesimal  region  5 A,  greater  is  the  resolution. 


The  solid  angle  subtended  at  the  entrance  pupil  of  the  lens  is  given  by: 

5  AC  os3  (ip) 


8d  = 


r- 


(28) 


where  /  is  the  focal  length  of  the  lens.  The  area  projected  onto  the  reflector  by  5 A  is  then: 

8ACos(ip)(d  +  z(t ))2 


5S  = 


(. f2Cos(<l> )) 


(29) 


Figure  12:  A  pixel  element  of  area  6 A  in  the  image  plane  projects  through  the  entry  pupil  of  the  lens  onto  the 
reflector  as  a  region  of  area  SS.  The  pupil  is  located  at  (0,  0,  d),  with  respect  to  the  origin.  The  principal  ray  from 
SA  reflects  off  the  reflector  at  sriixit,  9),y(t ,  9),  z(t ,  9)).  The  corresponding  viewpoint  on  the  caustic  surface  as 
as  shown  above.  The  solid  angle  subtended  at  this  viewpoint  is  then  8u>  =  dS/r2,  where  rc  is  the  distance  of  the 
viewpoint  from  the  reflector  (see  Eq.(6)).  Resolution  is  then  defined  as  a  ratio  of  bui  to  5 A. 


where,  0  is  the  angle  between  the  principal  ray  corresponding  to  5 A  and  the  surface  normal  at  SS.  The 
foreshortened  area  visible  to  the  viewpoint  (of  the  principal  ray)  on  the  caustic  surface  is  therefore: 

SACos(^)(d  +  z(t))2  (3()) 


The  solid  angle  subtended  at  this  viewpoint  is  then  given  by: 


Su 


5ACos{ip){d  +  z(t ))2 

P  ■  rc(t)2 


(31) 


where,  rc(t)  is  the  distance  of  the  viewpoint  from  the  reflector  surface.  The  resolution  for  any  rotationally 


symmetric  catadioptric  camera  is  then  given  by: 


SA  =  f2  •  r c(t)2 

5u i  Cos{p)(d  +  z(t))2 


(32) 


Substituting  Eq.7  in  Eq.32  we  obtain  the  expression  for  resolution  of  a  rotationally  symmetric  conic 
catadioptric  system. 


We  now  present  the  resolution  characteristics  for  some  configurations  of  conic  catadioptric  cameras. 
These  consist  of  a  conic  reflector  (whose  profile  is  a  conic  section)  and  a  perspective  lens  system,  whose 
entrance  pupil  is  located  at  a  finite  distance  from  the  reflector.  The  resolution  results  presented  are  only 
for  rotationally  symmetric  systems. 


Figure  13  illustrates  the  resolution  across  a  radial  slice  of  the  imaging  plane.  The  curves  have  been 
normalized  with  respect  to  magnification  such  that  area  of  the  reflector’s  image  in  all  three  cases  are  the 
same.  This  facilitates  a  fair  comparison  of  resolution  between  the  three  catadioptric  systems.  As  seen, 
resolution  drops  drastically  beyond  some  distance  from  the  image  center.  This  characteristic  gradually 
changes  as  entry  pupil  approaches  the  focal  point  of  the  reflector  (system  becomes  single  viewpoint).  In 


(a)  (b) 


Figure  13:  Plots  of  resolution  for  catadioptric  sensors  having  a  perspective  lens  based  camera  and  an  elliptic,  a 
parabolic  and  an  hyperbolic  reflector.  The  plots  illustrate  the  resolution  across  a  radial  slice  of  the  image  plane 
when  the  pupil  is  located  at  (a)  the  origin  ( d  =  0)  and  (b)  at  d  =  6. 


this  configuration,  resolution  increases  radially,  in  keeping  with  [2]. 

We  now  give  an  intuitive  explanation  as  to  why  resolution  tends  to  drop  towards  the  edge  of  the  caustic. 
Typically,  the  caustic  approaches  the  reflector  surface  radially  outwards.  Eventually  the  caustic  touches 
the  reflector  at  the  “tangent  point”.  Thus,  rc  — >  0  and  hence  the  solid  angle  subtended  increases,  reducing 
resolution.  In  contrast  with  single  viewpoint  systems,  rc  increases  as  we  move  radially  outwards,  thus 
increasing  resolution. 

By  careful  choice  of  reflector  and  positioning  of  the  lens,  we  can  take  advantage  of  the  initial  rise  in 
resolution  (see  figure  13(b) ).  For  instance,  by  zooming  into  the  reflector  (at  the  cost  of  reduced  field  of 
view)  we  can  use  only  the  section  of  the  reflector  for  which  the  optical  resolution  increases. 

5.2  Single  Reflector  with  Telecentric  Lens 


We  now  discuss  a  catadioptric  imaging  system  consisting  of  a  telecentric  lens  and  a  single  conic  reflector. 
From  Fig.  14,  projection  of  the  area  5 A  in  the  imaging  plane  onto  the  reflector  is  5A/Cos{4>).  The 
foreshortened  patch  which  is  visible  from  the  viewpoint  on  the  caustic  surface  is  5 A.  The  solid  angle 
subtended  at  this  viewpoint  is  given  by: 


(See00  = 


521 

^•oo2  ’ 


(33) 


where  rc  is  the  distance  of  the  viewpoint  from  the  point  of  reflection  on  the  reflector.  If  A4  is  the 
magnification  factor,  resolution  of  such  a  sensor  is  given  by: 


5A 

5u°° 


=  M  ■  r 


oo2 


(34) 


For  conic  reflectors,  the  distance  of  the  viewpoint  from  the  point  of  reflection  is  given  by: 

{t{p  +  (-1  +  e2)t){2p  +  (-1  +  e2)t)) 


r„  = 


(2  P2 


(35) 


Then,  resolution  for  a  conic  catadioptric  imaging  system  with  a  telecentric  lens  is: 

SA  ( t{p  +  (-1  +  e2)t)(2p  +  (-1  +  e2)t))2 

5u>°°  4 p4 


(36) 


We  now  apply  results  derived  in  Eq.(36)  to  conic  catadioptric  cameras  fitted  with  a  telecentric  lens  and 
study  their  resolution  characteristic s(see  Fig.  15). 


Imaging  plane 


Figure  14:  A  pixel  element  of  area  5 A  in  the  image  plane  projects  through  the  telecentric  lens  (orthographic 
projection)  onto  the  reflector  as  a  region  of  area  SS.  The  principal  ray  from  5 A  (corresponding  to  the  center  of 
the  area  5 A)  reflects  off  the  reflector  at  sn(x(t.  9),y(t ,  6 ),  z(t.  0)).  The  corresponding  viewpoint  on  the  caustic 
surface  is  as  shown  above.  The  solid  angle  subtended  at  this  viewpoint  is  then  5ui  =  SS/r where  rc  is  the 
distance  of  the  viewpoint  from  the  reflector  (see  Eq.(6)).  Resolution  is  then  defined  as  a  ratio  of  Scu  to  5 A. 

For  parabolic  reflectors,  the  caustic  reduces  to  a  single  point  (see  [1,  2]).  Thus,  the  resolution  increases 
radially  as  expected.  The  caustic  surface  for  an  elliptical  reflector  with  telecentric  lens  is  similar  in 
nature  to  that  with  a  perspective  lens.  Hence,  the  resolution  characteristics  are  also  similar,  exhibiting  an 
initial  rise  and  then  a  drop  towards  the  periphery.  Finally,  the  hyperbolic  reflector  yields  a  caustic  which 
is  expanding  in  nature  (see  figure  5  (i) ).  Also,  the  direction  of  expansion  is  in  the  direction  opposite  to 
that  of  the  point  of  reflection.  This  results  in  radially  increasing  resolution. 


Figure  15:  (a)  Resolution  across  a  radial  slice  of  the  image  plane  by  a  catadioptric  sensor  consisting  of  a  telecen- 
tric  lens  and  a  parabolic,  elliptic  and  hyperbolic  reflector  respectively.  The  resolution  plots  have  been  normalized 
to  illustrate  the  manner  in  which  resolution  changes  across  the  image  plane,  (b)  For  elliptic  reflector  based  cata¬ 
dioptric  cameras,  the  resolution  initially  rises  to  a  maxima  then  drops  to  zero  at  the  periphery  of  the  field  of  view. 
This  drop  to  zero  corresponds  to  the  point  on  the  reflector  at  which  the  caustic  surface  touches  it  (see  Eq.(21)). 
Flowever,  this  is  not  the  case  with  parabolic  or  hyperbolic  reflectors. 


5.3  Designing  Resolution  Specific  Sensors. 

The  analytic  expressions  for  resolution  of  catadioptric  cameras  are  not  restricted  to  conic  reflectors  alone. 
In  general,  the  same  framework  can  be  extended  to  any  mirror  shape.  Moreover,  the  formulae  derived 
above  not  only  aid  in  the  analyzing  the  resolution  of  the  sensor,  but  can  also  be  used  to  drive  its  design. 

Consider  the  class  of  conic  reflector  based  catadioptric  systems.  Resolution  is  parameterized  by  the 
geometry  of  the  reflector,  the  location  of  the  entrance  pupil  and  the  focal  length  of  the  lens  used.  Given 
an  apriory  resolution  curve,  we  can  “fit”  the  right  parameters  in  the  model  (e,  p,  d,  /),  that  most  closely 
approximate  the  required  curve.  In  the  most  generic  setting,  we  could  let  resolution  characteristics 
completely  dictate  reflector’s  shape  (not  restricted  to  conic  reflectors).  It  should  however  be  noted  that 
by  fixing  resolution  the  sensor  may  not  maintain  a  single  viewpoint.  Depending  on  the  application  at 
hand,  this  may  or  may  not  be  critical. 


6  Self  Calibration  of  Non-Single  Viewpoint  Cameras. 

So  far  we  discussed  the  geometry  and  resolution  characteristics  of  conic  catadioptric  cameras.  We 
derived  a  parametric  expression  for  their  viewpoint  loci  in  terms  of  the  geometry  and  location  of  the 
various  imaging  components.  If  the  exact  geometry  and  configuration  of  the  imaging  components  such 
as  lenses  and  mirrors  is  known,  the  caustic  or  viewpoint  locus  can  be  derived  analytically.  However, 
when  these  parameters  are  unknown,  self-calibration  is  required.  Although  every  pixel  has  an  unique 
viewpoint  and  viewing  direction,  the  caustic  formulation  reduces  the  complexity  of  calibration  to  that  of 
estimating  only  a  handful  of  parameters. 

In  the  past,  techniques  for  calibrating  single  viewpoint  catadioptric  systems  have  been  suggested  using 
a  single  image  [11]  or  camera  motion  [18].  Recently,  Grossberg  and  Nayar  suggest  using  caustics  to 
represent  generic  imaging  systems,  and  also  presented  a  calibration  technique  which  requires  known 
motion  as  well  as  an  active  display  of  known  light  patterns  (see  [13]).  In  contrast,  our  technique  cali¬ 
brates  non-single  viewpoint  systems  having  a  parametric  form  for  the  caustic,  using  only  known  camera 
motion  and  point  correspondences  between  views  of  unknown  scene  points  by  estimating  the  model 
parameters. 

6.1  Objective  Function  Formulation. 

We  pose  the  caustic  estimation  problem  as  one  of  error  minimization.  As  shown  in  Fig.  16,  let  p1  and 
p2  be  the  images  of  a  static  scene  point  P,  in  the  two  views.  From  the  hypothesized  parameter  values 
(during  search),  we  map  pi  and  p2  to  their  corresponding  viewpoints  (Si  and  S2)  as  well  as  their  viewing 
directions  (Vx  and  V2).  Camera  motion  is  not  the  only  factor  that  contributes  to  a  new  viewpoint.  Since 
the  image  is  formed  by  reflection  off  a  curved  surface,  the  second  view  is  reflected  off  a  different  point 
on  the  reflector.  This  maps  to  a  different  position  on  the  caustic  as  illustrated  in  Fig.  16.  Since  the  two 
light  rays  come  from  the  same  scene  point,  they  must  intersect  at  the  scene  point.  This  happens  when 
the  hypothesised  caustic  parameters  match  that  of  the  system. 

In  general  we  can  define  the  objective  function  as  the  distance  between  these  to  rays.  However,  we  found 
it  more  stable  to  define  the  error  function  in  the  image  space.  To  do  so,  we  first  derive  in  closed  form, 
a  point  P  along  Vx  which  is  closest  to  ray  V2  (this  is  the  best  estimate  of  the  scene  point  P).  This 


Figure  16:  The  catadioptric  imaging  system  is  represented  here  by  only  its  caustic  surface  (viewpoint  locus). 
The  two  caustic  surfaces  denote  the  same  camera  after  being  given  a  known  translation  T.  An  unknown  scene 
point  P  visible  from  both  views  gets  imaged  at  two  point  p\  and  p2  respectively.  From  the  caustic  model  and  the 
hypothesized  parameters  during  search,  we  map  these  image  point  to  their  respective  viewpoint  S\  and  S2  and 
their  viewing  directions  V\  and  V2.  When  the  right  model  parameters  arc  estimated,  the  two  position  vectors  (light 
rays  at  viewpoints)  must  meet  at  the  scene  point. 

hypothesized  scene  point  P  is  then  mapped  onto  the  image  plane  in  view  two,  using  the  caustic  model. 
We  then  define  the  error  as  the  disparity  between  the  image  of  P  in  the  second  view  p2  and  known  image 
in  the  p2: 

e  =  \p2  ~  P2I2  (37) 

The  objective  function  is  defined  over  all  the  pairs  of  point  correspondences  between  the  two  views. 
Thus,  if  pi1  refers  to  the  ith  image  point  in  view  j,  then  the  objective  function  for  N  scene  points  is  given 
by: 

Z=  J2\P2-P2%  (38) 
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Figure  17:  (a)  A  catadioptric  camera  consisting  of  a  perspective  lens  based  digital  camera  and  a  spherical  reflector 
(ball-bearing).  The  reflector’s  radius  is  25.4mm,  and  lens’  entry  pupil  is  approximately  150mm  above  the  center 
of  the  sphere.  The  catadioptric  camera  was  placed  on  a  translation  stage  and  translated  by  20mm  along  the  Y  axis, 
(b)  Sample  image  acquired  during  the  calibration  process.  Each  image  is  2048  x  1536  in  size. 

6.2  Calibrating  a  Real  Sensor 

We  now  present  calibration  results  on  a  real  non-single  viewpoint  catadioptric  sensor.  The  sensor  con¬ 
sists  of  a  perspective  lens  based  high  resolution  digital  camera  and  a  spherical  reflector  (ball  bearing). 
The  reflector  was  two  inches  in  diameter  and  was  placed  approximately  150mm  below  the  digital  camera 
(see  Fig.  17(a)). 

The  catadioptric  camera  was  placed  rigidly  on  a  translation  stage  to  facilitate  accurate  camera  motions. 
Between  acquiring  the  two  views,  the  sensor  was  translated  precisely  by  20mm  sideways  (along  the  Y 
axis).  Figure  17(b)  shows  a  typical  image  acquired  by  this  catadioptric  sensor. 

Since  the  parameterization  in  Section  2  is  singular  for  a  sphere,  we  parameterize  the  reflector  surface  as: 

A  ■  z2  +  y2  +  B  ■  z  =  C,  (39) 

which  includes  the  entire  class  of  conic  reflectors.  As  before,  the  entrance  pupil  is  at  a  distance  d  from 
the  origin  along  the  Z  axis.  The  caustic  surface  is  then  described  by  a  four-parameter  (A,  B,  C,  d )  family 
of  curves  (see  Appendix  A). 
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Figure  18:  Estimated  and  ground  truth  parameters  for  the  catadioptric  camera  shown  in  Fig.  17.  The  estimation 
was  done  using  constrained  minimization  routines  in  Matlab. 

Since  the  algorithm  depends  on  the  non-single  viewpoints  to  estimate  the  model  parameters,  it  is  imper¬ 
ative  that  the  chosen  point  correspondences  come  from  near  by  scene  points.  Just  like  in  stereo,  with 
greater  depth,  parallax  effects  diminish,  thus  minimizing  image  disparity  produced  by  the  the  viewpoint 
locus.  In  our  experiment  we  used  41  scene  points  (image  correspondences)  to  guide  the  non-linear 
search.  Since  we  know  the  reflector  to  be  spherical  we  only  need  to  estimate  the  focal  length  /  of  the 
perspective  lens,  the  distance  d  of  the  lens’s  entrance  pupil  from  the  reflector,  and  the  radius  of  the 
reflector  \[C-. 

Figure  18  provides  a  comparison  between  the  estimated  model  parameters  and  ground  truth.  The  ground 
truth  is  based  on  careful  manual  measurement  of  the  required  parameters.  The  recovered  radius  of  the 
reflector  is  accurate  to  within  a  few  millimeters.  The  estimated  focal  length  too  is  close  to  the  ground 


Estimated  viewpoint  locus 


Figure  19:  Ground  truth  (solid  curve)  and  estimated  caustic  (dotted  curve)  for  the  imaging  system  shown  in 
Fig.  17.  As  seen,  the  error  in  the  estimated  viewpoint  locus  of  the  camera  is  marginal  and  follows  the  ground  truth 
caustic  closely. 


truth.  However,  the  estimate  of  d  is  a  little  erroneous.  This  can  be  either  due  to  incorrect  convergence  of 
the  search  or  from  erroneous  measurement  of  ground  truth.  Once  the  model  parameters  are  estimated  we 
can  plot  the  viewpoint  locus  for  the  estimated  caustic  profile  (dotted  curve)  against  ground  truth  (solid 
curve)  (see  Fig.  19). 

7  Summary 

For  a  while  it  has  been  considered  necessary  for  imaging  sensors  to  maintain  a  single  viewpoint.  In 
reality  this  limits  the  flexibility  on  has  in  designing  optical  systems.  For  instance,  we  know  only  conic 
reflectors  provide  single  viewpoint  catadioptric  systems.  Thus,  one  can  use  only  elliptical,  parabolic 
or  hyperbolic  sensors.  Also,  such  imaging  systems  require  precise  alignment  of  the  various  imaging 
components.  Any  miss-alignment,  and  the  system  deviates  strongly  from  a  single  viewpoint. 

If  we  move  away  from  the  single  viewpoint  constraint,  then  we  have  much  more  flexibility  in  designing 
imaging  systems.  For  instance,  we  are  no  longer  limited  to  using  only  conic  reflectors.  More  exotic 
reflectors  could  be  used  to  develop  new  sensors.  This  paper  presents  a  step  towards  understanding 
such  non-single  viewpoint  imaging  systems.  We  focussed  on  conic  reflectors  not  only  because  most 
catadioptric  systems  utilize  such  reflectors  but  also  because  locally  any  surface  can  be  approximated  by 
quadrics. 

We  derived  a  concise  analytic  expression  for  the  loci  of  viewpoints  (caustics)  for  catadioptric  imaging 
systems  utilizing  conic  reflectors.  In  such  sensors,  every  pixel  in  such  systems  has  an  unique  viewpoint. 
Using  the  framework  of  caustics,  the  viewpoint  surface  can  be  parameterized  with  a  small  number  of 
parameters.  These  parameters  depend  on  the  surface  geometry  of  the  reflector  and  the  relative  location 
of  the  entrance  pupil  of  the  lens  with  respect  to  the  reflector.  Under  this  generic  framework,  the  single¬ 
viewpoint  catadioptric  camera  was  also  shown  to  be  a  special  case  of  the  general  solution.  In  general, 
any  imaging  system,  whose  components  (lenses  and  reflectors)  are  completely  known,  can  be  replaced 
by  the  effective  viewpoint  surface  for  the  purposes  of  analysis. 

Apart  from  deriving  an  analytic  expression  for  the  viewpoint  surface,  we  also  studied  other  geometric 
properties  of  the  sensor.  For  instance,  it  was  shown  that  the  cusp  formed  along  the  optical  axis  for 


rotationally  symmetric  systems  was  closely  related  to  approximating  the  entire  viewpoint  surface  by  a 
single  point.  We  also  proved  that  the  caustic  associated  with  the  tangential  light  ray  to  the  reflector  lies  at 
the  point  of  tangentiality.  This  point  “tangent  point”  was  solved  for  in  closed  form  giving  a  good  bound 
on  the  size  of  the  viewpoint  surface. 

An  important  reason  to  stray  away  from  a  single  viewpoint  is  to  design  imaging  systems  with  specific 
resolution  characteristics.  We  provide  a  simple  analytic  study  of  resolution  for  such  camera  systems.  We 
showed  how  resolution  degrades  radially  beyond  some  distance  from  the  image  center,  if  the  effective 
viewpoint  locus  is  not  a  single  point.  However,  careful  design  of  the  imaging  system  in  terms  of  lens 
and  reflector  choice  and  their  relative  positioning,  can  be  used  to  exploit  the  initial  rise  or  plateau  effect 
on  resolution.  Further  the  analytic  form  for  resolution  can  used  to  optimize  the  design  parameters  of  the 
imaging  system  for  apriori  resolution  requirements. 

Finally,  we  presented  a  simple  calibration  technique  to  estimate  the  viewpoint  surface  and  camera  param¬ 
eters  for  a  conic  catadioptric  system  using  known  camera  motion.  In  general  the  problem  of  calibrating 
non- single  viewpoint  cameras  is  that  of  estimating  a  map  between  each  pixel  in  the  image  and  its  unique 
viewpoint  and  viewing  direction.  Using  the  framework  of  caustics  this  potentially  hard  problem  was 
reduced  to  that  of  estimating  a  small  number  of  model  parameters.  Although  we  assumed  knowledge 
of  the  reflector  shape  (conic  cross-section),  the  precise  geometry  of  the  reflector  and  camera  as  a  whole 
was  unknown  and  was  estimated  from  two  images  of  an  arbitrary  scene. 


Appendix  A 

We  now  derive  the  caustic  surface  for  a  parameterized  conic  reflector  given  by: 

Az2  +  7  2 +  Bz  =  C.  (40) 

This  parameterization  not  only  includes  parabolic  (c  =  0,  A  =  0),  elliptic  ( B  =  0)  and  hyperbolic 
(A  <  0,  C  <  0)  reflectors,  but  also  spherical  (A  =  1  ,  C  +  >  0).  As  opposed  to  Eq.2  which  didn’t 

model  spherical  reflectors,  Eq.40  includes  the  sphere  as  well. 

In  parametric  form,  Eq.  40  can  be  rewritten  as: 

z(t)  =  t 

7(f)  =  y/C-Bt-  At 2  (41) 


Using  the  Jacobian  method  [7]  we  derive  the  caustic  profile  as  a  parametric  curve: 


ScNx(t) 


ScNyit ) 
ScD(t) 


Sc'(t ) 


B3C 2  +  6B2Cd  -  4 BC2d  +  8 Bd2  -  8 Cd2  -  3 (B  -  2 A2C)(B2C  +  4 Bd  - 

6 (B3  -  2(1  +  A2)B2C  +  2A2(1  +  A2)BC 2  +  4A2(A2  -  1  )Cd)t2  +  4((2 A2 

-A2(  1  +  3A2)BC  +  2A4(1  +  A2)C2  +  AA2(A2  -  1  )d)t3, 

A(B 2  -  ABC  —  Ad  +  AA2(C2  +  d))(d  -  t(B  +  A2t))^3/2) 

-B3C  +  AB2C2  +  2 B2d  -  ABCd  +  16 A2C2d  -  8 d2  +  16 A2d2 

-3 {B3  +  2 (A2  -  2 )B2C  +  A(2A2  -  1  )Bd  -  8A2Cd)t 

-12A2(A2  -  1  )(BC  +  2d)t2  +  AA2(A2  -1  )(B  -  2 A2C)t3, 
fSxNx(t )  SxNy{t )1 
1  S'cD(f)  ’  5cU(f)  ^ 


ACd)t  + 

-  l)^2, 


(42) 


Equation  (42)  describes  the  caustic  profile  for  the  complete  class  of  conic  (including  spherical)  catadiop- 
tric  cameras.  In  calibrating  such  non-single  viewpoint  imaging  systems,  when  the  reflector  shape  is  not 
known,  we  should  use  the  above  derived  model. 
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